Method and device for recognising distance in real time

ABSTRACT

A device for recognizing distance in real time includes first, second, and cameras. The third camera arranged nearer the first camera than the second camera. The first, second and third cameras acquire simultaneously first, second and third images, respectively, and an electronic circuit of the device estimates the distance of an object as a function of a stereoscopic correspondence established between first and second elements representative of the object. The first and second elements belong to the first and second images, respectively. The stereoscopic correspondence is established by a relationship between the first elements and corresponding third elements belonging to the third image.

TECHNICAL FIELD

The present invention relates to the distance recognition field and morespecifically according to the stereoscopic measurement technique basedon two cameras intercalated by a third guiding camera.

PRIOR ART

Generally speaking, the stereoscopic measurement technique consists inusing two images taken by two cameras spaced apart, in order todetermine the distances of objects taken in the image. The distances arecomputed by matching the contents of the two images.

However, a stereoscopic system with two cameras is easily made defectiveby masked objects or repetitive structures such as for example avertical grid. Indeed, the two cameras may lead to confusion between forexample two vertical bars of the grid.

In order to resolve in part this type of problem, it is known from otherstereoscopic measurement techniques using more than two cameras toduplicate the viewpoints and the potential matchings. Such an example isdescribed in the document CA2245044 which concerns stereovisioncomputations carried out on different pairs of cameras. The differentcomputations are compared in order to reject poor matchings. Anotherexample is described in the document JP561240376 which uses threefocusing systems to avoid stereovision errors when for example a thinobject is situated behind another and then appears inverted between thetwo images.

All these techniques make the basic resolution cumbersome with a quitelow gain and necessitate complex algorithms requiring great computingpower.

The aim of the present invention is to propose a method and a device forrecognising distance in real time, overcoming the aforesaid drawbacks,in particular by resolving the problems due to repetitive structures ormasked objects while not having to resort to complex computationalgorithms.

DESCRIPTION OF THE INVENTION

The present invention is defined by a device for recognising distance inreal time including first and second cameras, comprising:

-   -   a third camera arranged nearer the first camera than the second        camera, said first, second and third cameras being configured to        acquire simultaneously first, second and third images        respectively, and    -   an electronic circuit configured to estimate the distance of an        object as a function of a stereoscopic correspondence        established between the first and second elements representative        of said object and belonging to the first and second images        respectively, said stereoscopic correspondence being established        by taking into account a relationship between said first        elements and corresponding third elements belonging to the third        image.

This device makes it possible to facilitate the determination ofrelationships between the contents of images while having goodresolution for the recognition of distances, and to do so while beingcapable of not allowing itself to be tricked by repetitive structures ormasked objects. Indeed, the fact that the third camera is very close tothe first camera means that the shift between the contents of the thirdand first images is very small making it possible to reduce thecorrespondence search zone and consequently to determine a very preciseapplication between the contents of the first and third images.Moreover, the further away cameras make it possible to increase theresolution of the estimation of distances of the different objects.Thus, a distance in stereovision in real time is made without having toresort to complex computation algorithms.

Advantageously, the electronic circuit is configured to extract from thefirst, second and third images first, second and third elements selectedfrom the following elements: contours, image segments, and rectangularimage sub-zones.

This makes it possible to minimise the amount of information on theimages thereby facilitating the configuration of the electronic circuitwhile maintaining distance recognition precision. It will be noted thata contour corresponds to a pixel of an image line where the change inluminous intensity is maximum compared to its neighbouring pixelssituated on the same line (such as for example the edge of a white stripof a pedestrian crossing). The image sub-zones are two-dimensional zones(for example squares of 5×5 pixels of a same line).

Advantageously, the third camera is arranged at a predetermined distancefrom the first camera, said predetermined distance being configured sothat each first element in the first image is associated with a singlethird element in the third image. For example, it is advantageous thatthe third camera is situated at 1 cm, or even less, from the firstcamera.

This facilitates the establishment of correspondences between thecontours of the images captured by the near cameras.

It will be noted that the quotient of the distance between the first andthird cameras on the distance between the first and second cameras maybe smaller than or equal to around one fifth. This optimises the ratiobetween the simplicity and the precision of the device.

Advantageously, the first, second and third cameras are arranged in aco-linear manner and the electronic circuit is configured to compare thefirst, second and third contours belonging respectively to the first,second and third images in an independent manner and line by line.

In an alternative, the first, second and third cameras are aligned andare configured to capture the first, second and third imagesunidimensionally. Thus, it suffices to have a single 1D image bar (i.e.a line of pixels).

Advantageously, the electronic circuit is configured to:

-   -   extract the first, second and third elements belonging to a same        horizontal line on the first, second and third images,    -   establish an application between the first elements and the        third elements by associating for each first element a single        corresponding third element comprised in a reduced intermediate        search zone belonging to the third image, and    -   establish said stereoscopic correspondence between the first        elements and the second elements by associating for the single        third element comprised in the corresponding reduced        intermediate search zone a single corresponding second element        comprised in a second search zone belonging to the second image,        said second search zone being able to contain several contours.

This makes it possible to eliminate uncertainties of relationshipsbetween the contents of the first and second images.

According to a preferred embodiment of the present invention, saidfirst, second and third elements are first, second and third contoursrespectively.

According to this embodiment, the electronic circuit comprises:

-   -   contrast extraction filters configured to carry out a        convolution on a same line of each of the first, second and        third images thereby forming first, second and third contrast        curves on said same line of first, second and third images,    -   thresholding filters configured to carry out thresholdings on        said first, second and third contrast curves thereby forming        first, second and third contours respectively,    -   encoders configured to carry out an encoding on said first,        second and third contours thereby forming first, second and        third discrete levels respectively, and    -   a circuit of (basic) electronic components configured to:        -   determine a first pixel shift between the position of each            first contour and that of the corresponding third contour            thereby defining said application between the first contours            and the third contours,        -   determine a second pixel shift between the position of each            first contour and that of the corresponding second contour            thereby defining said stereoscopic correspondence between            the first contours and the second contours, and        -   estimate the distance of an object as a function of the            second pixel shifts between the positions of the first            contours and those of the second contours representing said            object.

Thus, the device comprises very simple electronic operations that aresummed up in basic operations on individual pixels of a line of theimage without even searching for shapes in all of the pixels of thecomplete 2D image.

Advantageously, the electronic circuit comprises an alert mechanismconfigured to signal any object as soon as its distance with respect tothe cameras begins to reach a predetermined lower limit.

Advantageously, the first, second and third cameras are visible lightcameras or infrared cameras. This enables the device to operate in aluminous or dark environment.

Advantageously, the device comprises a pattern projector configured toproject repetitive patterns facing the first, second and third cameras,said repetitive patterns having a spacing greater than that between thefirst and third cameras.

Advantageously, the electronic circuit further comprises a processorconfigured to carry out a rectification of the images before carryingout the distance recognition.

This makes it possible to correct potential alignment problems betweenthe different images.

Advantageously, the processor is further configured to carry out a shaperecognition on image zones identified according to a single distance.

Advantageously, the processor may also be configured to process theelements corresponding to image segments or rectangular image sub-zonesby making them more discernible or more relevant.

Advantageously, the processor is configured to estimate an approximatedistance on an image without contours by linking up pixels having anidentical luminous intensity.

According to another embodiment of the invention, the device furthercomprises a series of additional guiding cameras arranged according toincreasing gaps going from the first camera to the third guiding camera.

This makes it possible to use cameras with a very high number of pixelsand to work in zones with very close contours.

According to a particular embodiment, the device comprises an additionalguiding camera, the gap between the first camera and the third camerabeing around one fifth of that between the first camera and the secondcamera, and the gap between the first camera and the additional guidingcamera being around a twenty fifth of that between the first camera andthe second camera.

According to yet another embodiment of the invention, the devicecomprises a plurality of other cameras arranged along differentdirections with respect to the co-linear arrangement of the first,second and third cameras.

Advantageously, the device comprises fourth and fifth cameras spacedapart by a predetermined distance and mounted perpendicularly withrespect to the co-linear arrangement of the first, second and thirdcameras, the fourth camera being arranged near to the first camera, saidfirst, second, third, fourth and fifth cameras being configured toacquire images simultaneously.

The invention also relates to a mobile system comprising the deviceaccording to any of the preceding claims.

Thus, the mobile system (example vehicle, flying object, drone) equippedwith this device is capable of recognising obstacles in real time and ofhaving an overview of the environment without using very ponderouscomputations.

The invention also relates to a method for recognising distance in realtime using first and second cameras, and comprising the following steps:

-   -   arranging a third camera near to the first camera, said first,        second and third cameras being configured to acquire        simultaneously first, second and third images respectively, and    -   estimating the distance of an object as a function of a        stereoscopic correspondence established between the first and        second elements representative of said object and belonging to        the first and second images respectively, said stereoscopic        correspondence being established by taking into account a        relationship between said first elements and corresponding third        elements belonging to the third image.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

FIG. 1 schematically illustrates a device for recognising distance inreal time, according to an embodiment of the invention;

FIG. 2A illustrates an image taken by the recognition device, accordingto the invention;

FIG. 2B illustrates a representation of the image taken by therecognition device, according to the invention;

FIG. 2C illustrates another image taken by the recognition device,according to the invention;

FIG. 2D illustrates a representation of the other image taken by therecognition device, according to the invention;

FIG. 3 schematically illustrates a device for recognising distance inreal time, according to a preferred embodiment of the invention;

FIG. 4A schematically illustrates a method for recognising distance inreal time, according to an embodiment of the invention;

FIG. 4B illustrates a luminous intensity curve used in the method ofFIG. 4A, according to an embodiment of the invention;

FIG. 4C illustrates a contrast curve used in the method of FIG. 4A,according to the invention;

FIG. 4D illustrates curves used in the method of FIG. 4A, according toan embodiment of the invention;

FIG. 4E is a covariance curve applied a first of the curves illustratedin FIG. 4D, according to an embodiment of the invention;

FIG. 4F is another covariance curve applied to a second of the curvesillustrated in FIG. 4D;

FIG. 4G is a further covariance curve applied to a third of the curvesillustrated in FIG. 4D;

FIG. 4H is a thresholded waveform used in the method of FIG. 4A,according to an embodiment of the invention;

FIG. 4I is a discretized waveform generated by the method of FIG. 4A,according to an embodiment of the invention;

FIG. 4J is another discretized waveform generated by the method of FIG.4A, according to an embodiment of the invention;

FIG. 5 schematically illustrates first, second and third encodedcontours shifted with respect to each other and derived from the first,second and third images taken simultaneously by the three cameras,according to an embodiment of the invention;

FIG. 6 schematically illustrates an electronic circuit according to apreferred embodiment of the invention;

FIGS. 7 and 8 schematically illustrate devices for recognising distancein real time, according to other preferred embodiments of the invention;and

FIG. 9 schematically illustrates a mobile system comprising a distancerecognition device, according to a preferred embodiment of theinvention.

DETAILED DESCRIPTION OF PARTICULAR EMBODIMENTS

The basic concept of the invention consists in adding an additionalcamera very close to one of the two cameras of a conventionalstereovision system while using a basic electronic circuit of lowelectrical consumption to determine the distance of an object. Theelectronic circuit is a hardware circuit of logic gates which makes itpossible to reduce the electrical consumption 10 to 100 times withrespect to that of a device of the prior art which uses a lot of complexalgorithms to avoid false correspondences.

FIG. 1 schematically illustrates a device for recognising distance inreal time, according to an embodiment of the invention. FIG. 1 alsoillustrates a method for recognising distance in real time, according toan embodiment of the invention.

The distance recognition device 1 comprises first C1, second C2 andthird C3 cameras as well as an electronic circuit 3.

The first C1, second C2 and third C3 cameras are mounted on a support 5and are laid out to acquire simultaneously first I1, second I2 and thirdI3 images respectively. Advantageously, the cameras C1, C2, C3 supplydigital images I1, I2, I3 composed of a determined number of pixelsdefining the resolution of the images.

The third camera C3 is a guiding camera situated between the two camerasC1 and C2 and, more specifically, arranged very close to one of them.According to this example, the third camera C3 is arranged near thefirst camera C1. Thus, the distance d1 separating the first camera C1from the third camera C3 is less than the distance d2 separating thesecond camera C2 from the third camera C3. For example, the distance d1is five to ten times smaller than the distance d2.

The fact that the first C1 and third C3 cameras are very close meansthat the images I1 and I3 that they capture are very similar to within asmall shift of several pixels, the shift being linked to the distance ofthe observed object. This shift is of zero pixels if the object is atinfinity and of the order of several pixel units to several tens ofpixels (depending on the resolution of the image) if it is very near tothe cameras C1, C2, C3. This small shift delimits the search forcorrespondence between shapes contained in the two images I1 and I3 to asmall search zone. This makes it possible to very easily link thecorresponding shapes of these two images I1 and I3.

It will be noted that the pixel shift between the images I1 and I2captured by the extreme cameras (i.e. first C1 and second C2 cameras) isgreater than that between the images I1 and I3 captured by the nearcameras (i.e. first C1 and third C3 cameras) by a factor that depends onthe quotient between the distance d3 of the extreme cameras C1, C2 andthe distance d1 between the near cameras C1, C3.

Advantageously, the quotient between the distance d1 separating thefirst C1 and third C3 cameras on the distance d3 separating the first C1and second C2 cameras is less than or equal to around a fifth. Forexample, in the case where this quotient is equal to a sixth (i.e. thedistance between C1 and C2 is six times greater than that between C1 andC3), then a shift of “120” pixels between shapes contained in the firstI1 and second I2 images (having a resolution of several hundreds ofpixels) results in a shift of only “20” pixels between correspondingshapes contained in the first I1 and third I3 images. It will be notedthat a large shift (“120” pixels on a resolution of only several hundredpixels) may create an uncertainty on the matching of shapes contained inthe first I1 and second I2 images and notably for repetitive shapes suchas a grid or foliage. However a shift of only “20” pixels makes itpossible to elaborate a precise correspondence between the shapescontained in the first I1 and third I3 images. Thus, the third camera C3plays a guiding role which facilitates the determination of acorrespondence relationship between the shapes contained in the imagesI1 and I2 of the extreme cameras C1 and C2.

Moreover, the electronic circuit 3 is configured to establish astereoscopic correspondence between the first E1 and second E2 elementsbelonging to the first I1 and second I2 images respectively by takinginto account a correspondence relationship between the first elements E1and equivalent third elements E3 belonging to the third image I3.

Advantageously, the third camera C3 is situated at a predetermined (forexample, 1 cm) distance d1 from the first camera C1 such that a firstelement E1 in the first image is associated with a single third elementE3 in the third image.

It will be noted that the elements E1, E2 and E3 may be contours, imagesegments or image sub-zones, for example image sub-zones each having arectangular shape. The contours correspond to the edges of the objectsrepresented in the images whereas the image segments or sub-zonescorrespond to geometric shapes of rectangular, square type or othershapes, comprising a determined number of pixels (for example, squaresof 5×5 pixels).

To construct the stereoscopic correspondence A3 between the first E1 andsecond E2 elements belonging to the first I1 and second I2 images, theelectronic circuit 3 is configured to compose two applications. A firstapplication A1 is established between the first elements E1 belonging tothe first image I1 and the corresponding third elements E3 belonging tothe third image I3. A second application A2 is established between thethird elements E3 belonging to the third image I3 and the correspondingsecond elements E2 belonging to the second image I2. The images (in themathematical sense of the term, that is to say, the results) of thefirst application A1 are used as arguments for the second applicationA2.

Thus, the accuracy of the correspondence relationship between the firstI1 and third I3 images established by the application A1 enables theelectronic circuit 3 to estimate with precision the distance D of anobject R with respect to the device 1 as a function of the stereoscopiccorrespondence A3 between first E1 and second E2 elements representativeof the object and belonging respectively to the first I1 and second I2images.

It will be noted that a conventional stereoscopic measurement techniquemay be used by the electronic circuit 3 to estimate the distance of anobject as a function of the pixel gap between the first E1 and second E2elements representative of the object. A conventional stereoscopicmeasurement algorithm is for example the FW (Fixed Window) methodaccording to a cost aggregation strategy. Such a method is described inthe document of Stephano Mattocia entitled “Stereo Vision: Algorithmsand Applications”. The operating defects of this conventional method areresolved by a reduced search zone thanks to the third camera accordingto the present invention.

Advantageously, the resulting image is displayed on an interface and thedifferent distances D of the different objects with respect to thedevice 1 may be represented on the resulting image according to adifferent colour scale as represented in FIG. 2.

Indeed, FIGS. 2A and 2C illustrate images taken by the recognitiondevice 1 and their representations are illustrated in FIGS. 2B and 2D,respectively. The representations illustrated in FIGS. 2B and 2D modelthe distances of the different elements determined by the electroniccircuit 3 according to a colour scale. The right hand scale indicatesthe correspondence between the luminosity of the contour and itsdistance, the lightest/most luminous colour being representative of acontour of the object situated at the smallest distance from the device1.

FIG. 3 schematically illustrates an example of a device for recognisingdistance in real time, according to a preferred embodiment of theinvention.

The distance recognition device 1 comprises first C1, second C2 andthird C3 cameras mounted on a support 5, taking for example the shape ofa guide strip. The first C1 and second C2 cameras are mounted on theends of the guide strip 5 whereas the third camera C3 is mounted nearerthe first camera C1 than the second camera C2. Advantageously, the firstC1, second C2 and third C3 cameras may be digital visible light orinfrared cameras. It will furthermore be noted that these cameras mayquite simply be sensors aligned and configured to capture first, secondand third unidimensional images.

The recognition device 1 also comprises an electronic circuit 3including first 31, second 32 and third 33 encoding devices connected tothe first C1, second C2 and third C3 cameras respectively as well as awiring circuit 34 including basic electronic components. The encodingdevices 31, 32, 33 are suited to reducing electronic noise, to searchingfor contours and to eliminating very poorly marked contours. Each of theencoding devices 31, 32, 33 comprises a contrast extraction filter 41,42, 43, a thresholding filter 51, 52, 53 and an encoder 61, 62, 63.

According to this example, the distance d3 between the first C1 andsecond C2 cameras is chosen to be around six times greater than thedistance d1 between the first C1 and third C3 cameras. This latterdistance d1 may be of the order of several centimetres, for example ofthe order of one to five centimetres. The digital resolution of thecameras may be of the order of several hundreds to several tens ofthousands of pixels. The objectives of the three cameras C1, C2 and C3are co-linear and, consequently, an element representing an object atinfinity bears the same number of pixels on the three images. However,the shift with respect to the first image of an element, taken by thefirst camera C1, representing a close object will be around six timesgreater on the second image than on the third image.

FIGS. 4A-4J schematically illustrate a preferred embodiment of a methodfor recognising distance in real time, according to the device of FIG.3.

Step S1 concerns the taking of the first, second and third images by thefirst C1, second C2 and third C3 cameras respectively at a same instant.Indeed, FIG. 4A illustrates as an example and schematically first I1,second I2 and third I3 images taken simultaneously by the three camerasC1, C2, C3 on which are represented first E1, second E2 and third E3elements shifted with respect to each other. According to this example,the elements correspond to images of objects such as a pedestriancrossing and a tree (see also FIG. 2C).

At steps S2-S5, the electronic circuit 3 is configured to extract fromthe first I1, second I2 and third I3 images first, second and thirdcontours respectively. Thus, according to this embodiment, the elementsare contours which represent the edges of objects.

Advantageously, the electronic circuit 3 is configured to analyse thefirst I1, second I2 and third I3 images line by line. Thus, at step S2 asame horizontal line number 1, is selected on the three images I1, I2,I3 (see FIG. 4A). For example, firstly, it is the first line of pixels,situated the lowest, of the images that is selected. Step S2 isrepresented in FIG. 4B which illustrates a curve of the luminousintensity as a function of the pixel number on the selected horizontalline l₁. For example, the first pixel is that situated the most to theleft of the line of pixels. For reasons of simplification, only thecurve relative to the first image is illustrated in FIG. 4B.

At step S3, the contrast extraction filters 41, 42, 43 are configured tocarry out a convolution on the same line 1, of each of the first I1,second I2 and third I3 images thereby forming corresponding first,second and third contrast curves CS1. FIG. 4C illustrates as an examplethe first contrast curves CS1 belonging to the first image I1. Moreover,the contrast curve extraction technique is illustrated in greater detailin FIGS. 4D-4G. The first graph of FIG. 4D illustrates an initial curveF1 representing the luminous intensity of a part of a line as a functionof the number of pixels. This corresponds for example to a zone of thegraph illustrated in FIG. 4B of step S2. In order to smooth the initialcurve, a first convolution COV1 (FIG. 4E) is applied to this initialcurve F1 using a Gaussian type low pass filter thereby forming asmoothed curve F2 illustrated in the second graph.

Next, the position of a change of contrast (i.e. contour position) isdetermined by computing the slope between two points sufficiently spacedapart. This may be done by applying a second convolution COV2 (FIG. 4F)to the smoothed curve F2 forming a contrast curve F3. As an example,Dirac functions are used to estimate the position of each contour.

In an alternative, the first and second convolutions may be groupedtogether into a single convolution COV3 (FIG. 4G) using for example awavelet of sinusoidal shape. It will be noted that the convolutionconsists in applying in a known manner a matrix of coefficients to awindow comprising a given pixel and its neighbours then sliding thiswindow over the whole image.

At step S4, the thresholding filters 51, 52, 53 are configured to carryout thresholdings on the first, second and third contrast curves CS1according to a predetermined rejection threshold S (see FIG. 4C). Thisrejection threshold S makes it possible to eliminate noise and smallvariations. The resulting first, second and third contrast curves(hereafter called first, second and third contours or peaks P) have goodcontrast definition. The result of step S4 is illustrated as an examplein FIG. 4H which shows the first resulting contours (or peaks) P of thefirst image I1.

At step S5, the encoders 61, 62, 63 are configured to carry out adiscrete encoding according to three output levels “−1”, “0” or “1” onthe first, second and third contours forming respectively first, secondand third discrete levels N (see FIGS. 41-4J). More specifically, thelevel “−1” is attributed to the negative peaks, the level “1” isattributed to the positive peaks and finally, the level “0” isattributed to the intervals between the peaks.

In an alternative, the first, second and third contours are encoded in abinary manner according to two values “0” or “1” by taking quite simplythe absolute value of the three preceding levels. Thus, the value “1” isattributed to both the positive peaks and to the negative peaks whereasthe value “0” is as previously attributed to the intervals between thepeaks. The two alternatives of step S5 are illustrated in FIGS. 41-4J.The discrete (−1, 0, 1) or binary (0, 1) values designate contours orpeaks and consequently, the terms “discrete value” and “contour” arequite simply merged hereafter.

At steps S6-S8, the wiring circuit 34 is configured to compare thepositions of the different contours P on the three images in order todetermine a correspondence between the contours P on the first I1 andsecond I2 images using the third image I3. This correspondence iscarried out from the discrete levels received line by line and pixel bypixel from the first 31, second 32 and third 33 encoding devicesconnected to the first C1, second C2 and third C3 cameras respectively.This correspondence next makes it possible to estimate the distance ofthe object.

In order to explain steps S6-S8, FIG. 5 schematically illustrates first,second and third encoded contours shifted with respect to each other andderived from the first, second and third images taken simultaneously bythe three cameras, according to an embodiment of the invention. It willbe recalled that an encoded contour represents the steepest slope ofchange of intensity between for example the grey of the tar and thewhite of the pedestrian crossing. It will furthermore be noted that thecontours in FIG. 5 may represent elements according to the presentinvention (i.e. contours, image segments or sub-zones).

The first contours P11-P13 derived from the first image I1, the secondcontours P21-P23 derived from the second image I2, and the thirdcontours P31-P33 derived from the third image I3 are shifted withrespect to each other but represent a same image element E1, E2, E3 ofthe same close object taken according to different viewpoints at a sameinstant. A first contour P11 on the first image I1 is associated with acorresponding third contour P31 which may be found in an intermediatesearch zone (designated guiding zone) Z1 of the third image I3. Thesecond contour P21 on the second image I2 corresponding to this firstcontour P11 is to be searched for in a large correspondence search zoneZ2. The description relative to the search zones Z1, Z2 will be detailedhereafter in the description.

According to this example, the large correspondence search zone Z2 inthe second image I2 comprises two distinct second contours P21 and P22consequently being able to induce uncertainty on the choice of thesecond contour. However, the guiding zone Z1 of the third image I3 onlycomprises a single third contour P31 which makes it possible toestablish a precise correspondence with the first contour P11 associatedwith the first image I1 and by composition of correspondences tocorrectly determine the corresponding second contour P21. Thus, thanksto a reduced extent of the guiding zone Z1, the electronic circuit 3 canestablish a precise stereoscopic correspondence between the first I1 andsecond I2 images.

Indeed, at step S6, the wiring circuit 34 is configured to establish anapplication between the first contours P11-P13 and the third contoursP31-P33 by associating for each first contour a single correspondingthird contour comprised in an intermediate search zone Z1 belonging tothe third image I3. More specifically, the wiring circuit 34 establishesthis application by determining a first pixel shift between the positionof each first contour P11-P13 and that of the corresponding thirdcontour P31-P33.

At step S7, the wiring circuit 34 is configured to establish thestereoscopic correspondence between the first contours P11-P13 and thesecond contours P21-P23. The single third contour P31 comprised in thecorresponding intermediate search zone Z1 is associated with a singlecorresponding second contour P21 comprised in a second search zone Z2belonging to the second image I2.

More specifically, the wiring circuit 34 establishes the stereoscopiccorrespondence by determining a second pixel shift between the positionof each third contour P31-P33 and that of the corresponding secondcontour P21-P23. This relationship corresponds to the second applicationbetween the third contours and the second contours.

At step S8, the wiring circuit 34 is configured to estimate the distanceof an object with respect to the device 1 as a function of the secondpixel shifts between the positions of the first contours P11-P13 andthose of the second contours P21-P23 representing the object inquestion. The estimation of the distance results from a predeterminedcorrespondence between distance values in centimetres or in metres andthe shift in number of pixels. This predetermined correspondence dependson the focal distance of the objectives of the cameras and the densityof the pixels. For example, if a same object situated at 1 m from thedevice 1 is shifted by one hundred pixels between the first C1 andsecond C2 cameras, then at 2 m it would have been shifted by fiftypixels, and so on up to zero pixels if it is at infinity.

Steps S6-S8 are described in greater detail with reference to FIG. 6 inaddition to FIG. 5.

Indeed, FIG. 6 schematically illustrates a more detailed electroniccircuit according to a preferred embodiment of the invention.

In order to simplify the illustration of the electronic circuit, theresolution of the cameras C1, C2, C3 is assumed to be of the order ofonly several hundreds of pixels. Moreover, the distance d1 between thefirst C1 and third C3 cameras is chosen quite small so that a very closeobject (for example at 1 m from the cameras), only has at the most ashift of six pixels between the contours representing them on the firstI1 and third I3 images knowing that an object at infinity does not haveany shift between the two images. It will be noted that the shift numberalso depends on the minimum contour recognition distance (for example at1 m from the camera) and on the resolution of the image in addition tothe distance between the two cameras. As an example, the distancebetween the first C1 and third C3 cameras is of the order of 5 cm andthat between the first C1 and the second C2 camera is of the order of 30cm. This enables the recognition device 1 to recognise objects between 1m and infinity.

With reference to FIG. 5, it is assumed that a first contour P11 isreferenced by a pixel number “n” on the first image I1. The thirdcontour P31 corresponding to this pixel “n” is found in the guiding zoneZ1 of the third image I3 defined by an interval I₁=[n, n+5]. For adistance between the first C1 and second C2 cameras six times largerthan that between the first C1 and third C3 cameras, the shift number inthe second image I2 may be comprised in an interval I₂=[n, n+28]representing the second search zone Z2. Here, the example is taken of amaximum shift for a close object, of five pixels between the firstcamera C1 and the third camera C3. Thus, even if the real shift would be5.5 pixels, it is seen all the same with five pixels. The second cameraC2, being five times further away than the third camera C3, should givethe contour corresponding to the furthest with 27.5 pixels (i.e. 5 times5.5 pixels). Consequently, the contour the most shifted between thefirst and second cameras C1, C2 will be with twenty eight pixelsmaximum. This makes it possible to cover the case where a contoursituated between two pixels on the third camera C3 appears randomlyeither in n+i or in n+(i−1) thereby constantly guaranteeing theexistence of a correspondence between the first and second cameras C1,C2.

Thus, according to this embodiment, the wiring circuit 34 illustrated inFIG. 6 comprises first 71 and second 72 horizontal wirings connected tothe outputs of the first 31 and third 33 encoding devices respectivelyas well as a first vertical wiring 73 connected to the output of thesecond encoding device 32. The wiring of the circuit is describeduniquely as an example and obviously may be configured differently bythose skilled in the art.

The first 71 and second 72 horizontal wirings represent comparison linesbetween the images of the first C1 and third C3 cameras (i.e. the nearcameras). Moreover, the outputs of this comparison are coupled to thefirst vertical wiring 73 to compare them with the image of the secondcamera C2.

At each tick of the clock, the first horizontal wiring 71 is configuredto receive the level of a single pixel of the first image. This level iseither a “1” representing a contour or a “0” representing the absence ofcontour. On the other hand, the second horizontal wiring 72 isconfigured to have already received the levels of five consecutivepixels of the third image I3 whereas the first vertical wiring 73 isconfigured to have already received the levels of twenty eightconsecutive pixels of the second image I2. Thus, each clock tickrelative to the first horizontal wiring 71 corresponds to the fifth andtwenty eighth clock ticks respectively relative to the second horizontalwiring 72 and to the vertical wiring 73. Thus, when the level of thepixel numbered “n” of the first image I1, noted p₁(n), is found on thefirst horizontal wiring 71, the levels of six pixels numbered from “n”to “n+5” of the third image I3, noted p₃(n), . . . , p₃(n+5) are foundon the second horizontal wiring 72 and the levels of twenty nine pixelsnumbered “n” to “n+28” of the second image I2, noted p₂(n), . . . ,p₂(n+28) are found on the vertical wiring 73.

Indeed, the second horizontal wiring 72 comprises five horizontal shiftoperators 75 (i.e. electronic shift elements or components) connected inseries making it possible to have the values of six consecutive pixelsof the third image I3. The level of the pixel p₃(n) of the third imageI3 is cascade shifted at each clock tick (relative to the secondhorizontal wire 72) by the horizontal shift operators 75 and is replacedby the level of the following pixel p₃(n+1). In other words, the pixelthat was on the left is displaced by the shift operator 75 to the rightat each clock tick. Thus, the levels of six consecutive pixels p₃(n), .. . , p₃(n+5) are found on the second horizontal wiring 72 separated twoby two in an alternating manner by each of the five shift operators 75.

Given that the guiding zone Z1 relative to the third camera C3 extendsbetween the pixels p₃(n), . . . , p₃(n+5), then the value (1 or 0) ofthe pixel p₁(n) (relative to the first image I1) on the first horizontalwiring 71 may thereby be compared with the values of the pixels p₃(n), .. . , p₃(n+5) (relative to the third image I3) on the horizontal secondwiring 72. This comparison is carried out by six horizontal “AND” logicgates PH0, . . . , PH5 coupling the first 71 and second 72 horizontalwirings. More specifically, the values of the pixels p₁(n) and p₃(n) areinjected into the horizontal logic gate PH0, the values of the pixelsp₁(n) and p₃(n+1) are injected into the horizontal logic gate PH1 and soon.

The shift between a contour P11 of the first image I1 and acorresponding contour P31 of the third image I3 is then determined bythe rank of the logic gate which has an output value equal to “1”. Forexample, if PHj=1, then it is known that the pixel shift betweencorresponding contours of the first and third images is equal to “j”.This relationship between the first contours representative of an objectof interest of the first image I1 and the corresponding third contoursof the third image I3 forms a first application between these two imagesrelative to this object of interest.

Furthermore, the first vertical wiring 73 comprises twenty eightvertical shift operators 76 connected in series making it possible tohave the levels (1 or 0) of twenty nine consecutive pixels of the secondimage I2. The level of the pixel p₂(n) of the selected line of thesecond image I2 is cascade shifted at each clock tick by the verticalshift operators 76 and is replaced by the level of the following pixelp₂(n+1). In other words, the pixel which was on top is displaceddownwards at each clock tick. Thus, the levels of twenty nineconsecutive pixels p₂(n), . . . , p₂(n+28) are found on the verticalwiring 73 separated in a sequential and alternating manner by the twentyeight vertical shift operators 76.

Given that the second correspondence search zone Z2 relative to thesecond camera C2 extends between the pixels p₂(n) and p₂(n+28), then theoutput levels of these pixels are compared with the outputs of the sixhorizontal logic gates PH0, . . . , PH5. More specifically, the levelsof the pixels on the vertical wiring 73 are compared by packets with theoutput values of the horizontal logic gates PH0, . . . , PH5.Advantageously, an overlap of two pixels is provided between twoconsecutive packets.

This comparison is carried out by vertical “AND” logic gates PV0, . . ., PV28 coupling the vertical wiring 73 with the outputs of thehorizontal logic gates PH0, . . . , PH5. More specifically, according tothis example, a first packet of seven pixels p₂(n+28), . . . , p₂(n+22)of the vertical wiring 73 is compared with the output of the horizontallogic gate PH5 through a first group of seven vertical logic gatesPV28(1), . . . , PV22(1). A second packet of seven pixels p₂(n+23), . .. , p₂(n+17) of the vertical wiring 73 is compared with the output ofthe horizontal logic gate PH4 through a second group of seven verticallogic gates PV23(2), . . . , PV17(2) and so on. It will be noted thatthe levels of the pixels p₂(n+22) and p₂(n+23) are taken into account inthe two comparisons. Indeed, according to this example, the twenty ninepixels p₂(n), . . . , p₂(n+28) are sub-divided into six packets of sevenpixels each and with two overlaps of two pixels between each pair ofadjacent packets. The overlap between two pixels makes it possible totake into account the case where a contour may be more or less definedbetween two adjacent pixels. In this case, the outputs of the verticallogic gates PV23(1) and PV23(2) are injected into an “OR” logic gatePL23 and similarly, the outputs of the vertical logic gates PV22(1) andPV22(2) are injected into another “OR” logic gate PL22. The overlap zoneis chosen according to the ratio of distances between the two closestcameras.

Finally, twenty nine output lines L0, . . . , L28 are formed by theoutputs of the vertical “AND” logic gates (in the zones of non-overlap)and the outputs of the “OR” logic gates indicating the shift between thefirst and second cameras. Indeed, if a contour is detected between thefirst and third images, there will only be a single output line amongthe twenty nine output lines L0, . . . , L28 which is going to have thelevel “1”.

Thus, the shift value between a contour of the first image I1 and acorresponding contour of the second image I2 is determined by the rankof the output line Li which is found at level “1”. For example, if Li=1,then it is known that the pixel shift between corresponding contours ofthe first I1 and second I2 images is equal to “i”.

These comparisons thereby associate for each third contour comprised inthe search guiding zone Z1 a single corresponding second contourcomprised in the second search zone Z2 belonging to the second image I2then determining the stereoscopic correspondence between the firstcontours and the second contours.

Advantageously, the wiring circuit 34 comprises a final column of output“AND” logic gates PS0, . . . , PS28 suited to validate the existence ofa single correspondence between a contour on the first image I1 and asingle contour on the second image I2.

These output “AND” logic gates PS0, . . . , PS28 are coupled to theoutput lines L0-L28 such that each output “AND” logic gate PSi receivesthe value borne by the output line Li as well as the inverse of thevalue borne by the preceding output line L(i−1). However, the firstoutput “AND” logic gate PS0 receives the value “1” in addition to thevalue borne by the first output line L1.

In order to illustrate the operation of the wiring circuit 34, let usassume that the level of the pixel number “n” is equal to “1” (i.e.p₁(n)=1) signifying the presence of a contour at the position “n”.Moreover, let us assume that the corresponding contour in the thirdimage I3 is detected at the position “n+4” (i.e. p₃(n+4)=1). The valuesof the other pixels are thus at zero(p₃(n)=p₃(n+1)=p₃(n+2)=p₃(n+3)=p₃(n+5)=0) knowing that the distancebetween the first C1 and third C3 cameras is configured so that there isa single contour in the guiding zone Z1. Furthermore, let us assume thatthe first pixel having the value “1” in the second image I2 is the pixelnumber “21” (i.e. p₂(17)=p₂(18)=p₂(19)=p₂(20)=0 and p₂(21)=1, p₂(22)=0,p₂(23)=0) then the levels on the line L21 and all the other output linesLx are equal to “1” and “0” respectively. The output logic gate PS21 isthen found at the level “1” and consequently, the shift between thecontour of the first image I1 and that corresponding of the second imageI2 is equal to “21” and this remains valid even if another contourexisted between p₂(1) and p₂(16) or between p₂(24) and p₂(28). It willbe noted that it would have been necessary to recognise this othercontour as not appropriate via a complex algorithm if the third guidingcamera had not made it possible to consider it as non-significant.

Next, the distance is directly deduced from the shift value using apredetermined function (or a curve) of the distance as a function of theshift.

Advantageously, the electronic circuit 3 comprises an alert mechanism 81(see FIG. 3 or FIG. 6) configured to signal any object of which thedistance with respect to the cameras C1, C2, C3 begins to reach apredetermined lower limit. This lower limit may be configured dependingon the type of application and may for example be equal to 50 cm or 1 m.

According to an embodiment, the distance recognition device 1 comprisesa pattern projector 85 (see FIG. 3) configured to project repetitivepatterns facing the three cameras C1, C2 and C3. The pattern projector85 is for example a laser pointer with a kaleidoscopic tip. Furthermore,the projection may advantageously be carried out in the near infrareddomain.

It will be noted that the spacing of the repetitive patterns isconfigured to be greater than that between the first camera and thethird guiding camera. Thus, these artificial patterns are easilyidentified by the three cameras C1, C2 and C3 without any risk ofcorrespondence error. This enables the recognition device 1 to gainfurther in performance on uniform zones (i.e. without marked contours)or in not very luminous zones. Here again, it will be noted that withoutthe third guiding camera, the repetition of the patterns would havecreated numerous potential correspondences difficult to reject withoutresorting to a complex algorithm.

Advantageously, the electronic circuit 3 may also comprise a processor83 (represented in dotted lines in FIG. 3) configured to carry out apotential rectification of the images so that a horizontal line is thesame on the three images. In this case, the images at the output of thecameras C1, C2, C3 pass through the microprocessor which rectifies thembefore transmitting them to the encoding devices 31, 32, 33.

It will be noted that the mechanical mounting of the cameras on a rigidsupport (for example a guide strip) makes it possible to have analignment on the images of the three cameras. However, in the case wherethe cameras are not calibrated or if the quality of their objectives isnot optimal or if the mechanical mounting of the three cameras is notperfect, the lines of each camera may be at different levels or even notparallel. Thus, in order to correct this potential problem, theprocessor comprises a rectification software (known to those skilled inthe art) to correct the alignment of the lines between the differentimages.

Furthermore, in the case where the elements extracted from the imagesare image segments or rectangular image sub-zones, the processor 83 isfurther advantageously configured to process the image segments or therectangular image sub-zones by making them more discernible. Thisprocessing may be carried out in a known manner by weighting the pixelscorresponding to these elements with greater weight, or by varying theircolour or their luminosity, etc.

Advantageously, the processor 83 is further configured to estimate anapproximate distance on an image zone without contours by linking pixelshaving an identical luminous intensity and situated in a distance zone(i.e. shift) identical to the surrounding contours already found.

The processor 83 may also be configured to fill the spaces between thecontours of which the distances have been calculated. This may becarried out according to a known technique of continuity of distancesbetween near pixels.

The processor 83 may also be configured to carry out a shape recognitionon image zones identified according to a single distance by implementingknown shape recognition software for example to identify persons.

FIG. 7 schematically illustrates a device for recognising distance inreal time, according to preferred embodiment of the invention.

According to this embodiment, the distance recognition device 1comprises a series of additional guiding cameras C31, C32, . . . , C33arranged according to increasing gaps going from the first camera C1 tothe third guiding camera C3. Thus, the gap between each additionalcamera and that which precedes it varies in an increasing manner.

This makes it possible to use cameras with a very high number of pixels(for example greater than or equal to 4000 pixels) enabling very precisemeasurement of distances. Moreover, this makes it possible to workeasily in zones with very close contours (for example, tree foliage).

As an example, the distance recognition device 1 comprises a singleadditional guiding camera C31. Thus, the distance recognition device 1comprises four cameras C1, C2, C3, C31. For example, the gap d1 betweenthe first camera C1 and the third camera C3 may be around one fifth ofthat d3 between the first camera C1 and the second camera C2. In thiscase, the gap d31 between the first camera C1 and the additional guidingcamera C31 may be around one twenty fifth of that d3 between the firstcamera C1 and the second camera C2.

To give numerical values, the distance between the first camera C1 andthe additional guiding camera C31 may be equal to around 1 cm, thedistance between the first camera C1 and the third guiding camera may beequal to around 5 cm, and finally, the distance between the first cameraC1 and the second camera C2 may be equal to around 25 cm.

FIG. 8 schematically illustrates a device for recognising distance inreal time, according to yet another preferred embodiment of theinvention.

According to this other embodiment, the distance recognition device 101comprises first C1, second C2, third C3, fourth C4 and fifth C5 camerasconfigured to acquire images simultaneously.

As previously, the first C1, second C2 and third C3 cameras are mountedin a co-linear manner on a support or a first guide strip 5. The thirdcamera C3 is a guiding camera arranged nearer the first camera C1 thanthe second camera C2. However, the fourth C4 and fifth C5 cameras aremounted on a second guide strip 105 arranged perpendicularly to thefirst guide strip 5. The fourth C4 and fifth C5 cameras are spaced apartby a predetermined distance d4 and are perpendicular to the co-lineararrangement of the first C1, second C2 and third C3 cameras. The fourthcamera C4 is a guiding camera arranged nearer the first camera C1 thanthe fifth camera C5 and plays the same role as the third camera C3. Allof the first C1, fourth C4 and fifth C5 cameras are configured to detecthorizontal contours whereas all of the first C1, second C2 and third C3cameras are configured to detect vertical contours. This recognitiondevice 101 enables good recognition of vertical and horizontal contoursthereby further increasing the precision of distance measurements.

The recognition device 101 also comprises an electronic circuit 103formed of first 103 a and second 103 b circuits each being equivalent tothe circuit 3 described previously. Indeed, the first circuit 103 a isassociated with the first C1, second C2 and third C3 cameras to detectvertical contours whereas the second circuit 103 b is associated withthe first C1, fourth C4 and fifth C5 cameras to detect horizontalcontours.

According to yet other embodiments, the recognition device 101 maycomprise a plurality of cameras arranged along different directions withrespect to the co-linear arrangement of the first, second and thirdcameras.

FIG. 9 schematically illustrates a mobile system comprising a distancerecognition device 1, according to a preferred embodiment of theinvention.

The mobile system 121 may be a terrestrial vehicle (car, lorry, train,etc.), maritime (boat, ship, etc.), an aircraft (airplane, helicopter,etc.) or a drone. By being equipped with a distance recognition device1, the mobile system 121 is suited to recognise obstacles in real timeand to have an overview of the environment without using very ponderousand costly computations in terms of computation time and electricalconsumption.

The distance recognition device 1 may be mounted on the mobile system121 along a direction selected from the following directions: parallelto the direction of gravity, perpendicular to the direction of gravity,or oblique with respect to the direction of gravity.

A direction parallel to the direction of gravity makes it possible toobtain distance information from the horizontal contours. A directionperpendicular to the direction of gravity makes it possible to obtaindistance information from the vertical contours. Furthermore, an obliquedirection with respect to the direction of gravity makes it possible toobtain distance information from the horizontal or vertical contours.

Advantageously, the mobile system 121 may comprise several distancerecognition devices 1 mounted along several different directions.Furthermore, the mobile system 121 may be equipped with a distancerecognition device 101 according to the embodiment of FIG. 7.

The invention claimed is:
 1. A device for recognizing distance in realtime, comprising: first and second cameras, a third camera arrangednearer the first camera than the second camera, the gap between thefirst camera and the third camera being smaller or equal to one fifth ofthat between the first camera and the second camera, said first, secondand third cameras being collinearly arranged and being configured toacquire simultaneously first, second and third images respectively, andan electronic circuit configured to estimate a distance of an object asa function of a stereoscopic correspondence established between firstand second elements representative of said object and belonging to thefirst and second images respectively, said stereoscopic correspondencebeing established by composing a first application between the firstelements and corresponding third elements belonging to the third imageand a second application between the third elements and thecorresponding second elements, the results of the first application areused as arguments for the second application, the first, second andthird elements including at least one of contours, image segments, andrectangular image sub-zones.
 2. The device according to claim 1, whereinthe electronic circuit is configured to extract from the first, secondand third images said first, second and third elements.
 3. The deviceaccording to claim 1, wherein the electronic circuit is configured to:extract the first, second and third elements belonging to a samehorizontal line on the first, second and third images, establish anapplication between the first elements and the third elements byassociating for each first element a single corresponding third elementcomprised in a reduced intermediate search zone belonging to the thirdimage, establish said stereoscopic correspondence between the firstelements and the second elements by associating for the single thirdelement comprised in the corresponding reduced intermediate search zonea single corresponding second element comprised in a second search zonebelonging to the second image.
 4. The device according to claim 1,wherein said first, second and third elements are first, second andthird contours respectively.
 5. The device according to claim 4, whereinthe electronic circuit comprises: contrast extraction filters configuredto carry out a convolution on a same line of each of the first, secondand third images thereby forming first, second and third contrast curveson said same line of first, second and third images, thresholdingfilters configured to carry out thresholdings on said first, second andthird contrast curves thereby forming the first, second and thirdcontours respectively, encoders configured to carry out an encoding onsaid first, second and third contours thereby forming first, second andthird discrete levels respectively, and a circuit of electroniccomponents configured to: determine a first pixel shift between theposition of each first contour and that of the corresponding thirdcontour thereby defining said application between the first contours andthe third contours, determine a second pixel shift between the positionof each first contour and that of the corresponding second contourthereby defining said stereoscopic correspondence between the firstcontours and the second contours, and estimate the distance of an objectas a function of the second pixel shifts between the positions of thefirst contours and those of the second contours representing saidobject.
 6. The device according to claim 1, wherein the electroniccircuit is further configured to signal any object as soon as itsdistance with respect to the cameras begins to reach a predeterminedlower limit.
 7. The device according to claim 1, wherein the first,second and third cameras are visible light cameras or infrared cameras.8. The device according to claim 1, further comprising a patternprojector configured to project repetitive patterns facing the first,second and third cameras, said repetitive patterns having a spacinggreater than that between the first and third cameras.
 9. The deviceaccording to claim 1, wherein the electronic circuit further comprises aprocessor configured to carry out a rectification of the images beforecarrying out the distance recognition.
 10. The device according to claim1, further comprising a series of additional guiding cameras arrangedaccording to increasing gaps going from the first camera to the secondguiding camera.
 11. The device according to claim 10, further comprisingan additional guiding camera, the gap between the first camera and theadditional guiding camera being one twenty fifth of that between thefirst camera and the second camera.
 12. The device according to claim 1,further comprising a plurality of other cameras arranged along differentdirections with respect to the co-linear arrangement of the first,second and third cameras.
 13. The device according to claim 12, furthercomprising fourth and fifth cameras spaced apart by a predetermineddistance and mounted perpendicularly with respect to the co-lineararrangement of the first, second and third cameras, the fourth camerabeing arranged near to the first camera, said first, second, third,fourth and fifth cameras being configured to acquire imagessimultaneously.
 14. A mobile system comprising the device according toclaim
 1. 15. A method for recognizing distance in real time, comprising:using first and second cameras, using a third camera situated nearer thefirst camera than the second camera, the gap between the first cameraand the third camera being smaller or equal to one fifth of that betweenthe first camera and the second camera, said first, second and thirdcameras being collinearly arranged and being configured to acquiresimultaneously first, second and third images respectively, andestimating a distance of an object as a function of a stereoscopiccorrespondence established between first and second elementsrepresentative of said object and belonging to the first and secondimages respectively, said stereoscopic correspondence being establishedby composing a first application between the first elements andcorresponding third elements belonging to the third image and a secondapplication between the third elements and the corresponding secondelements, the results of the first application are used as arguments forthe second application, the first, second and third elements includingat least one of contours, image segments, and rectangular imagesub-zones.